ZAHRA FAYAZ ( Test Method )
Zahra Fayaz Test Method WHAT IS TEST METHOD? ''' Test method is a definitive procedure that produces a test result. Test can be considered as technical operation that consists of determination of one or more characteristics of a given product. often a test is part of an experiment. The test result can be qualitative (yes/no) ,categorical , or quantitative ( a measured value). It can be a personal observation or the output of a precision measuring instrument '''Distribution of new information ' ' The distribution of the new information must be processed and manipulated in order for the test taker to successfully complete a given test task. It divided into two parts: A) Compact B) Diffuse -new information which is distributed over a short time may be called: compact -new information which is distributed over a long time may be called: diffuse Example 1: listening to a lecture and then summarizing its content, requires keeping in mind information from the beginning, the middle and the end, it is an example of diffuse Example 2: in a group oral interview consisting of fairly rapid exchanges, with each of the participants speaking one or two sentences in turn, the new information in each utterance is likely to be compact. What is input? Input consists of the new information contained in a given test task, to which the test taker is expected to respond. With highly compact input, the test taker may have very little opportunity to negotiate meaning, or may have to negotiate meaning very quickly, and such performance will be very demanding of his competence. An example of this might be a highly speeded listening comprehension test in which test takers have little time to consider their answer to a question before the next question begins. Input in which the information is highly diffuse may also be very demanding, in that it requires the test taker to encode information, hold it in memory and then recall it for subsequent processing. Examples of this would be a task that required summarizing a lengthy report, or one in which the test taker has to answer questions based on a lecture. Input in which the test taker has adequate opportunity to negotiate meaning, either through interaction or through repeating or slowing down the input, is neither compact nor diffuse. An oral interview conducted at ‘normal’ conversational speed, for example, provides ample opportunity for the test taker to interact with the interviewer in a way that enables her to interpret the information in the discourse and relate it to information in the context. Similarly, in a reading test with an ample time allotment, the test taker can read at his own pace, and reread sections that he finds difficult. Types of information ' ' ''' The type of information in input and expected response can be classified along three dimensions: concrete /abstract , positive/negative , and factual/counterfactual. NOTE: Expected response : 1) is a part of the test method. 2) We can specify it through our test design. 3) The nature of the expected response is based on the assumption that the test taker will be submissive to the context control introduced by the specific test task. Input and expected response affect performance on language tests. - Input contains A)format of language B)nature of language - Expected response contains A)types of restrictions on response B)degree of restriction on response '' '' ''Abstract information'' : represent symbolic or linguistic. for example standard deviation , it using mathematical symbols ''' S= It is more difficult to represent this concept visually (as a labeled diagram of a normal curve) . ''Concrete information : ''is visual , auditory , tactile ''Negative information : ''Information can also vary in terms of the degree to which it is negative. Information presented with negative markers can vary in both the level and number of elements negated. The sentences, ‘I didn’t want John to go to the store’, and ‘I wanted John not to go to the store’, for example, both include a single negative element, but differ in the level of negation, with the matrix sentence containing the negative element in the former, while in the latter, the embedded sentence carries the negative element. The sentence, ‘I didn’t expect him not to understand , on the other hand , contains two negative elements, one in the matrix sentence and one in the embedded sentence. It could be hypothesized that the more negative information ’contained in the input, the more difficult it will be to process. ''Positive information : ''Information that includes no explicit negative elements or markers can be considered to be positive, as in ‘Fred and I have agreed to meet at the airport at 10 o’clock.’ ''Counterfactual information : ''is about conditions which are possible or probable in some alternative world, but not in the known, factual world (van Dijk 1977:79). Counterfactual sentences typically take the form of conditionals, such as: ‘If cows could jump over the moon, dogs might laugh’, and differ in the degree of similarity they imply between the alternative world and the factual world. The possible world in which cows fly and dogs laugh, however, would be quite different from the factual world. We might hypothesize that counterfactual information would be more difficult to process than factual information, and would be more difficult to interpret than those in which the factual and possible worlds are quite similar. Topic : ' Topic is generally understood as ‘what it is about’; it is the subject matter of the discourse. The topic of the input in tests is determined by the test writer, who typically chooses topics that she feels will be interesting and relevant to test takers. the topic of the response is expected to match, or be relevant to that of the input. For example, in a test in which a test taker listens to a short lecture followed by a question about the content of the lecture and is required to give a short spoken answer, the individual’s answer may be evaluated on its relevance to the topic of the lecture. As indicated above, with respect to the relationship between new information in the response and information in the input, responses in which the topic is unrelated to the topic of the input may make it difficult to score the response according to the criteria set by the test writer. In tests in which there is an interactive relationship between input and response, it is possible for the topic of the discourse to be determined jointly by the test administrator and the test taker. The topic of input has an important effect on test takers’ performance. if the topic is familiar to some test takers and not others, these individuals may have an unfair advantage , resulting in better performance.’ Clearly , we want to avoid this as a source of “ test bias “ . on the other hand , we also want the subject which is interesting and relevant to test takers. test writers may select topics that are very general, and thus entirely unengaging to test takers. One the other hand, topics that are expected to be engaging because they are interesting and relevant in favor of some test takers. One possible solution to this problem is the presentation of a fairly large number of topics, which could reduce serious bias. '''Genre: ' According to: 1) Hymes (1972a) the term ‘genre’ refers to ‘formal characteristics that are traditionally recognized’, giving as examples the categories of ‘poem, myth, tale, proverb. 2)Coulthard (1977) a genre is one type of stylistic structure for organizing sentences and utterances into larger units such as greeting , farewell and prayers. 3)Brown and Yule (1983) suggest that genres may differ in their formal characteristics such as paragraph structure and the distribution of sentences types. if the language of the input in a given test is characteristic of a genre that is unfamiliar to the rest taker, we would hypothesize that tasks that depend on the interpretation of that input would be relatively difficult. Unfamiliarity with the characteristics of a given genre may also make the expected response more difficult. 'Organizational characteristics : ' Organizational competence comprising those abilities that are related to controlling the formal organization of language. This formal organization is a characteristics of both the input and the response. It is in three types : 1)grammar 2)cohesion 3)rhetorical The input material is organized in terms of grammar , cohesion and rhetorical Structure, and in tests that require a selected response it may include errors involving one or more organizational characteristics. An example of input involving grammatical errors is a multiple-choice item in which several words in a sentence are underlined, one of which is ungrammatical. 'Pragmatic characteristics : ' The input and response can be characterized in the term of the language function performed. As noted by Searle (1969), the function performed by any ‘exam question’ is not to elicit information, but rather to find gut if the examinee knows something (p. 66). Input consisting of a language sample thus performs the ‘primary’ function of requesting a response. More important, input performs language functions in addition to the manipulative function that is inherent in the test. Input and the response can involve the performance of the variety of a language function.an example of this is a oral interview in which the examinee is involved in the conversation that hi virtually forgets the formal organizational characteristics of the discourse . Determining the functionality of a given test task is problematical, because it depends on the interaction between the test taker and the task material . A given task may be functional for some test takers and not for others. 'Sociolinguistic characteristic: ' It consist of : 1) Dialect or variety 2) Register 3) Naturalness 'Dialect or variety : ' While we need to test an individual’s ability to use different dialects or varieties of a given language appropriately, in the majority of language testing situations the target language to be measured is in some sense regarded as a ‘standard’ dialect or variety. We need to make every effort to ensure that the dialect or variety of language of both the input and the expected response is appropriate to the language use context in which test takers are most likely to find themselves using the language, or which is recognized by language educators as the most appropriate variety. For example, to the extent that the dialect of language used and taught in schools is different from students’ native regional. In India, for example it may be appropriate to use Indian English as the test language in some contexts and British English in others. In testing second or foreign language proficiency, the variety of language used in the test needs to be appropriate to the needs of the learners. Thus, for American college students studying Spanish, it may be that a variety of Spanish spoken in Latin America is appropriate, while Castilian Spanish might be more appropriate for students of Spanish in England. 'Register: (formal/informal) ' the register of test language needs to be appropriate to the context and needs of rest takers. In many tests, particularly those designed to be used in educational settings, the register of input and expected response is formal. When the register of the test language is not entirely appropriate to the register of the target language use context , the test results may provide misleading information. 'Language use : ' Bloom and Lahey (1987) divided language into three separate but overlapping components : content , form , use According to : 1) Function of language : language use, refers to the reasons why people communicate. 2) Context of language : language use , refers to how people both understand and choose from alternative linguistic forms in order to reach their goals. ''Restrictions on expected response : '' '' '' Virtually all language use is restricted in various way by the context or situation. For example, Upon meeting a colleague on the way to class, we are usually not free to utter whatever may come into our head, but are restricted our perform (greeting) and the register we use(formal, informal). The language used in language tests is sometimes characterized as ‘non-natural’, or ‘non-normal’, or relatively ‘artificial’, One way to characterize artificiality of language use is in terms of inappropriately restricted. There are five different areas in which restrictions can be placed on expected responses in language testing situations: (1) restrictions on the channel. (2) restrictions on the format. (3) restrictions on organizational characteristics. (4) restrictions on propositional and illocutionary characteristics. (5)restrictions on the time or length of the response. 'Restrictions on channel : ' Language use takes place under a wide variety of conditions. Conversations are carried out face-to-face, in quiet rooms, and on noisy streets. Phone conversations take place out of visual contact over lines. In fact, many types of language use normally take place under less than ideal conditions. for example, conversation with taxi driver. In many language testing situations, such interference in the channel does not occur and, therefore, cannot influence the performance of the participants. 'Restrictions on format : ' In some tests the format is highly restricted, as in selection or identification response types, while in other tests the format may be fairly unrestricted, as in a composition test requiring the production of a writing sample. 'Restrictions on organizational characteristics : ' It divided into two parts : 1) Restrictions on grammatical forms : Restrictions on form at the word level are common in language testing. The most obvious example is the multiple-choice vocabulary item in which test takers are required to deal with the meanings of a specific pair of words, without the opportunity to express the equivalent meaning in their own words. Restrictions on form at the sentence level occur in tests, where the materials are designed to elicit particular structures. 2) Restrictions on the organization of discourse : A high level of restriction on language performance is in the organization of discourse. An example of such a restriction would be requiring that a student follow a specific rhetorical pattern , such as comparing and contrasting two things. Another example is the oral interview , in which the subject is told to pretend to call up a close friend and converse with him on the telephone. the organization is restricted by instructing the subject to do four things: first, inquire about the friend’s health and the health of the members of his family; second, find out what the friend had been doing; third, suggest that the two of them do something together that evening; and finally, make plans for how to get where they have decided on going (Bachman and Palmer 1983a). 'Restrictions on propositional characteristics : ' Restrictions may also apply to the propositional content . consider the meaning of the following utterance: ‘How many children do you want to have?’ The propositional content is about family size. Consider the illocutionary force of this question, were it to be posed unexpectedly by a young man on his first date with a young woman. His intention might well be to increase the intimacy of the relationship. The effect, of course , might be to break up the relationship , if the woman were not at all interested in having children. 'Restrictions on time or length of response : ' Language use is always constrained , by time or length. In testing situations , administrative considerations almost place additional restrictions on time or length. ''Relationship between input and response : '' It can be classified into three types: 1) Reciprocal input and response 2) Nonreciprocal input and response 3) Adaptive input and response 'Reciprocal input and response : ' Reciprocal language use can be defined as the use of language by one individual to produce an effect in another individual through the reduction of uncertainty with knowledge of results. This definition contains a number of components. The term '''‘language’ focuses on the verbal (spoken or written) , as opposed to the non-verbal (for example, gestures, pictures) components of discourse. The phrase ‘'one' individual’ and ‘'another individual'’ make explicit the requirement that at least two parties - a sender and a receiver - be involved. The phrase''' ‘to produce an effect’''' indicates that the language use has a communicative goal. The phrase ‘reduction of uncertainty’ characterizes the means by which the communicative goal is effected, that is, by means of a change in information at the disposal of the receiver. One of the distinguishing characteristics of reciprocal language use, is interaction , the second distinguishing characteristic of reciprocal language use, is the presence of feedback, the test taker receives feedback on the correctness of his answers, an example would be oral interview, in which there is continual interaction between input and response. Nonreciprocal input and response : ' Nonreciprocal language use is that in which there is no interaction between language users, that is, in which continual give and take between two or more individuals does not take place. In nonreciprocal language use, therefore, there is no feedback. In reading a book, for example, the reader is seldom able to give feedback to the author, and the text of the book will not change. Other examples of nonreciprocal language use include letter writing , reading, listening to lectures, watching movies and talking to oneself. Nonreciprocal test tasks, are those in which both interaction and feedback are missing, as in a cloze or dictation test. language use in general is often nonreciprocal, as in reading books and newspapers or listening to radio and television programs. '''Adaptive input and response : ' If the input is influenced by the response, but without the feedback, then the relationship between input and response is adaptive. In an adaptive , the particular tasks to the test taker are determined by her response to previous tasks. For example an individual taking an adaptively administered multiple-choice test, typically begins with an item that is of medium difficulty. If she answers this item correctly, the next item presented will be slightly more difficult. In most adaptive tests, the test taker is not aware of the correctness of her responses. The difference between adaptive and reciprocal test is : Adaptive tests do not involve the feedback or reduction of uncertainty through knowledge of results that characterizes reciprocal language tests. An oral interview is reciprocal. multiple-choice items is not reciprocal, since the test taker may be totally unaware of the effect of her responses on the selection of test items, and receives no feedback on the correctness of her answers. 'In summary : ' Relationship between feedback: knowledge of interaction: response input and response: correctness of response: affects input: Reciprocal + (present) + (present) Adaptive - (absent) + (present) Nonreciprocal - (absent) - (absent) ' ' '' '' '' '